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ABSTRACT 

This paper discusses the competency space approach to 
diagnosing misconceptions, skill, and knowledge acquisition. In some 
approaches that combine misconceptions, skill, and kn iwledge 
acquisition, the latent ability theta is used more or less as an 
insignificant element, but in the competency space approach, a 
multidimensional latent space is used. The competency space approach 
is explained and an example is given of its use. It is necessary to 
expand the acceleration model to the multidimensional space if it is 
to be used effectively in the competency space approach. A slow and 
patient approach that attempts to identify the subspace of theta that 
deals with attribute mastery (domain knowledge) (theta b) of a 
relatively small dimensionality first may be productive. Once a 
unidimensional theta b has been identified, the acceleration model 
can be tested. The example given uses data from a trigonometry test 
for high school students in Japan. The competency approach can be 
effectively used in research in which an appropriate software is 
developed and used for intensive observation of an individuals 
problem-solving behavior. Two tables and seven figures illustrate the 
analysis. An appendix contains the test answers. (Contains 24 
references .) (SLD) 
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I. Introduction 

With a rapid progress of computer technologies, now we can program a well-controlled cogni- 
tive experiment in computer software, accomodate the software in a number of microcomputers, 
and have each trained experimenter carry one of the microcomputers and collect data, con- 
ducting the experiment on the individual basis. In this way, we can easily collect data for 
several hundred individual subjects without using too much time. With such a set of data not 
only we can observe each individual's behavior intensively but also observe his/her behavior in 
perspective, clarify individual differences, etc. 

Tatsuoka (1985, 1990) has developed a method for diagnosing misconceptions, skill and 
knowledge acqusition combining deterministic concepts of attribute and ideal response patterns 
with a latent trait 0 . 

DiBello, Stout and Roussos (1993) have proposed a model for cognitive assessment: 

P g (a,0) = (1 -pjft s ag P 9 (0 + Ac,) + (1 - d 9 )P 9 {6)) , 

where g denotes item, a is an attribute pattern, which is expressed as a sequence of k 
0's (— nonpossession) and l's (= possession), 9 is the latent residual ability, p is the slip 
probability, d g is the probability to select the strategy represented by the vector q g , for 
answering item g , which is expressed as a sequence of k 0's (= not needed) and l's (= 
needed), s Qg is the conditional probability of applying q g correctly, given that q g is selected, 
A is a constant usually set equal to 2.0, c g is a measure of completeness of q g , P g (0) is 
the probability of correctly applying all required attributes other than those indicated in the 
q g vector which follows the one-parameter logistic model (Rasch model), and P g (a,0) is the 
conditional probability of correct answer to item g , given a and 0 . 

In these approaches, the latent ability 0 is used more or less as an insignificant element. 
Samejima (1993a, 1994a) has proposed the competency space approach for cognitive assessment, 
in which a multidimensional latent space is used. Unlike other approaches, the competency 
space approach uses latent ability spaces agressively, making attribute mastery diagnosis easy 
and more precise, assessing ability to use mastered or partially mastered attributes, etc. 

II. Developmental Changes 

When our interest in assessing children's skills of addition, multiplication, fractions, neg- 
ative numbers, etc., mastery or non-mastery of these skills may cover all important aspects. 
Developmental psychology tells, however, that there are more attributes to acquire and use as 
one grows and becomes capable of handling more complex problems, and also each attribute 
becomes more complex. Thus attribute mastery itself becomes more complicated, and also ac- 
quisition of attributes and ability to identify necessary attributes, which have already mastered 
or partially mastered, structure them for solving a new problem, and use them innovatively, 
etc., will gradually be differentiated. As one becomes more mature and challenges intellectual 
tasks, such dynamics become more and more important. 
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To give an example, we come across graduate students who are excellent in course work, but 
cannot design dissertation research, which is essential for a Ph. D. candidate. Such individuals 
lack ability to identify the necessary attributes in, and retrieve them from, our long term 
memory, structure them well, etc. Unlike domain knowledge, it is more difficult to assess such 
dynamics and screen applicants for graduate programs in terms of this kind of ability before 
decision has to be made as to whether an applicant should be accepted or rejected. Thus 
cognitive diagnosis should deal with both of these two differentiated aspects, whereas this may 
not be very important when we deal with children. 

III. Competency Space Approach 

Competency space approach assumes that mastery of an attribute is a continuous process, 
although observable indices may be just mastery and non-mastery. Thus a continuous latent 
variable is hypothesized behind each attribute mastery. The approach is more or less focused 
on situations in which: 

1. Our target populations of individuals are on relatively high levels of intellectual ma- 
turity, and ability to use mastered attributes in innovative ways has become more 
important. 

2. Thus two subspaces are considered in the competency space, although a clear-cut 
separation may not be feasible. Let 0 denote the competency space, which is 
multidimensional. This competency space is decomposed into two subspaces, such 
that 

where the subspace 0 a deals with attribute masteries, or domain knowledge, and the 
subspace 0 6 concerns with ability to use the mastered attributes dynamically. 

It is conceived that, as one grows and becomes intellectually more mature, the roles of the 
subspace 0& will become more important. 

An advantage of the competency space approach is the possibility of reducing the dimension- 
ality of 0 , as factor analytic ideas indicate. Operationally, reduction of dimensionality can be 
done by transforming the manifest variables to the set of principal components, and discarding 
those components the variances of which are negligibly small; this can also be done by using the 
factor analysis model, discarding the unique factors. Note, however, that these procedures are 
justifiable only when the unique factor variances are all negligibly small. If some of them are 
large enough, thon i.hese unique factors should be included in determining the dimensionality 
of 0 a . 

It may be advisable to use both factor analysis and the principal component analysis. Factor 
analysis will tell how many unique factor variances are substantially large, and with this number 
added by the number of common factors becomes the estimated dimensionality of 0 . This 



number should be checked against the results of the principal component analysis, to find out 
if it agrees with the number of principal components whose variances are not negligibly small. 
If it does, then those components with negligibly small variances can be discarded; if it does 
not, include more components to make it agree. This is important for making assessment in 
0 a accurate. 

Once this has been done, the remaining principal components must be rotated to make each 
dimension meaningful. An appropriate method will be oblique pattern matrix rotation (e.g., 
Lawley k Maxwell, 1971), devising the pattern matrix to decompose 0 into the subspaces 
0 a and 06 . 

It should be noted that the estimated dimensionality of 0 partly depends on the homo- 
geneity of the target population. If individuals of the target population are educated fairly 
similarly, for example, the structure of the latent variables behind attribute masteries will be 
relatively simple. If the target population consists of people with variously different cultures, 
for example, the competency space will require a larger dimensionality. Thus cross-validation 
of the dimensionality is necessary, in order to make the usefulness of the operationally defined 
competency space broader. 

ASSESSMENT IN 9 fl 

Higher mental processes include not only more complicated attributes, but also a larger 
number of attributes. This will require more elaborate methodologies in assessing mastery 
or partial mastery of attributes in the subspace 0 a , and also with a powerful methodology 
cognitive assessment will become more precise and informative. 

Grades of Attainment 

Assessment in the subspace 0 a deals not only with the dichotomous categories of mastery 
and non-mastery but also partial mastery, with the introduction of the concept of the grade of 
attainment, in an effort to make the diagnosis more precise and informative. 

Take trigonometry, for example. There are many problem solving tasks which require some 
levels of attainment toward mastery of trigonometry, but net the complete mastery of trigonom- 
etry. For some problems, understanding sine, cosine and tangent on the triangle level without 
dealing with negative values (grade 1) may be sufficient. For some others, understanding those 
concepts on the triangle level with possibly negative values (grade 2) may be required, or those 
on the unit circle level using radians (grade 3) may be mandatory. 

Thus diagnosis will be made more precisely, and the number of attribute patterns, and hence 
that of non-fuzzy response patterns will increase. Table 1 presents sixteen conceivable states of 
attribute mastery when there are four attributes, A, B, C and D, on the left hand side. Since 
at least one attribute is needed to make an item, in this example there are fifteen conceivable 
types of items requiring one or more attributes. The non-fuzzy response patterns of these fifteen 
items corresponding to the sixteen states of attribute masteries are given on the right hand 
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side of Table 1. 



Insert Table 1 About Here 



If the grade of attainment is accomodated and the states of attribute mastery are described 
by 0 , 1 (grade 1 attainment) and 2 (mastery), then the number of conceivable states of 
mastery will be increased to 3 4 = 81 , and there will be eighty conceivable types of items. 
If a test includes all these types, then there will be the same number of non-fuzzy response 
patterns. Also the number of possible fuzzy response patterns becomes enormous. 

A strength of the competency space approach is that all these response patterns can be 
hadled without difficulty, by virtue of the graded response model (Samejima, 1969, 1972, 1994c). 
Thus there is no need to attempt to reduce the number of response patterns, nor to classify 
them into a smaller number of categories. 

The relationship between each grade of attainment toward mastery of an attribute and 0 a 
is probabilistic, rather than deterministic, with the reduction of the dimensionality of the latent 
space. This is illustrated by the curves in Figure 1, when 0 a turns out to be unidimensional. 



Insert Figure 1 About Here 



It is important to select a suitable model out of the family of graded response models 
(Samejima, 1972, 1994c), and criteria for this selection will be discussed later. Also as an 
example of suitable models for cognitive assessment, the acceleration model (Samejima, 1994c) 
will be introduced and discussed later. 

Attribute Diagnosis 

Attribute diagnosis in the sense of Tatsuoka (1985, 1990) and of DiBeilo, Stout and Roussos 
(1993) can be done practically in the same way in the competency space approach. 

Let g denote an attribute, and X g be a grade of attainment toward mastery of each 
attribute, and x g (= 0, l,.. M m ff ) denote its realization. The operating characteristic, P Xg (Qa) , 
of the grade x g means the conditional probability, given 0 a , with which the individual gets 
x y , that is, 

P Xg (e a ) = prob.[X g = x g | 0 fl ] . 

For a set of n (> 1) attributes, a response pattern, denoted by V , indicates a sequence of 
X g for 5 = 1,2, ...,n , and its realization, v , can be written as 

v = { *t Y • 

It is assumed that local independence (Lord & Novick, 1968) holds, so that within any group 
of individuals all characterized by the same value of 9 the distributions of x g 's are all 



independent of each other. (The acceptability of this assumption is determined when we decide 
on the dimensionality of 0 operationally.) Thus the operating characteristic, of the response 
pattern v is defined as 

P v (Q a ) = prob.[V = v | e a ] = II^ s ( 0 a) , 

which is also the likelihood function, L(v \ 0 a ) , for V = v . 

Using this likelihood function, the maximum likelihood estimate of the position of each 
non-fuzzy response pattern, as well as of each fuzzy response pattern, can be obtained. Thus 
an examinee with a given response pattern will be assigned to one of the non-fuzzy response 
patterns in terms of his/her state of attribute mastery, selecting one with the shortest distance 
in 0 a or based on the likelihood ratio, etc. 

Because of the introduction of the grade of attainment, this diagnosis can be done more 
precisely, that is, instead of deciding on either mastery or non-masiery of each attribute, the 
grade of attainment in the attribute in question is diagnosed for non-mastery individuals. 

ASSESSMENT IN 0 6 

The graded response model (Samejima, 1969, 1972, 1994c) takes essential roles in the as- 
sessment in the subspece 0& . If the number of observable steps in, say, problem solving, then 
the amount of information will be increased and the estimation of the subject's position in Qb 
will become more accurate. This is important, for screening graduate program applicants in 
terms of this aspect of suitability for a future Ph. D., for example, will be done with little risk. 

Suppose, for example, that a cognitive process, like problem solving, contains a finite or 
enumerable number of steps. For convenience, let g be a problem, and x g be a step. The 
graded item score x g should be assigned to the individuals who have successfully completed 
up to the step x g but failed to complete the step [x g + 1) . 

Let M Xg (Q{,) be the processing function (Samejima, 1994c) of the graded item score x g , 
which is the probability with which the individual completes the step x g successfully, under 
the joint conditions that: 

1. the individual's position is 06 , and 

2. the steps up to (x g - 1) have already been followed and completed successfully. 

It is assumed that M Xg (Q b ) is either strictly increasing in any dimension of Q b or constant, 
for x g = 1,2, ...,m g . This assumption is reasonable considering that each item has some direct 
and positive significance to the ability measured. 

Let (m g + 1) be the hypothesized graded item score adjacent to and above m g . Since 
everyone can at least obtain the item score 0 , and no one is able to obtain the item score 



(rrtg + 1) , it is reasonable to set 



= 1 for x g = 0 

= 0 for x g = nig + 1 



for all 06 . Thus the operating characteristic, P Xg (Qb) , of the graded item score x g is given 

by 

ft.(e*) = n ^«( 0 >) [i - M(*, +1) (0 6 )] . 

u<r ff 

This provides the fundamenta 1 . framework for the general graded response model. 

Let P* 9 (Ob) denote the conditional probability with which the individual of ability 0 6 
follows and completes the cognitive process successfully up to the step x g , or further. Thus 

p;,(Q b ) = n aw) . 

u<x g 

This function is called the cumulative operating characteristic (Samejima, 1994c), although 
cumulation is in the opposite direction. Thus the operating characteristic, P Xg (Qb) , can also 
be expressed by 

^(06) = p;,(0 6 ) - P( m§ +i){e>) . 

It is obvious that P* g (Qb) is also either strictly increasing in 0 6 or constant for all 06 , and 
assumes unity for x g = 0 and zero for x g = m g + 1 for the entire range of 0 6 . From this 
Px g (Qb) , the operating characteristic, ^(06 is obtained by 

p v (e b ) = P rob.[v = v | e 6 ] = n P *,( Q ») . 

which is also the likelihood function, L{v | 0 6 ) , for V = v . Thus the maximum likelihood 
estimate of the individual's position in 06 is obtained using this L(v \ Qb) • 

Since experiments consist of continuous discoveries, it will be wise to use a nonparametric 
approach for estimating the operating characteristics (e.g., Leveine, 1984; Samejima, 1983, 
1993a, 1994b) before fitting any parametric model. Figure 2 illustrates a nonparametrically 
estimated curve using simulated data, by the differential weight procedure of the conditional 
p.d.f. approach (Samejima, 1994b). Unlike the logistic curve, which was fitted to the theoretical 
curve and plotted in the same figure, the curve obtained by the differential weight procedure 
can disclose non-monotonicity of the theoretical curve. Thus for any identified pattern of 
behavior shared by a subgroup of individuals it will be better that the operating characteristic 
be estimated nonparametrically. 



Insert Figure 2 About Here 
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If no strong relationship with any dimension of the subspace 0j, is found out, that is, each 
and every curve turns out to be flat, the pattern of behavior is called diffused. Some possible 
interpretations are that the pattern of behavior under observation may have multiple meanings, 
it may be an indicator of the dimension 06 does not include and addition of a new dimension 
in 06 may be necessary, etc. In any case, search for the true reason is mandatory. 

If r strong relationship between the pattern of behavior and any dimension of the competency 
space is discovered, the pattern of behavior is said to be concentrated. The next step is to 
tentatively parameterize the nonparametrically estimated operating characteristics to make 
mathematical handling easy, using a semi-parametric models such as Ramsay & Wang's (1993). 
After this, some appropriate graded response model, such as acceleration model, can be fitted 
(cf. Samejima, 1994c). A similar approach can be used for any identified buggy pattern of 
behavior, which can also be useful in cognitive diagnosis. 

If we only have data of a limited number of subjects, we could replace the estimated operat- 
ing characteristic by the sum of the estimated conditional distributions of the latent trait, given 
the maximum likelihood estimates of the trait, in order to obtain some amount of information. 
These conditional distributions can be obtained in the processes of nonparametric estimation of 
the operating characteristics, such as the differential weight procedure and the simple sum pro- 
cedure of the conditional p.d.f. approach (Samejima, 1994b). Figure 3 illustrates concentrated 
and diffused patterns of behavior. 



Insert Figure 3 About Here 



INSTRUMENTS 

It may be feasible to categorize cognitive processes into the following three categories with 
respect to their assessabilities: 

1. Assessable by paper-and-pencil tests. 

2. Difficult to be assessed by paper-and-pencil tests, but assessable by computerized 
tests, using figural responses, etc. 

3. Not assessable by either of them, and intensive observations in experimental situations 
are needed. 



With advancement of computer technologies, category 2 will take more and more from 
category 3. Category 1 has advantage over the other two categories in the economy of time and 
cost, however. Since we cannot aford to conduct research which belongs to category 2 without 
a sizable research money, it may be wise to start with paper-and-pencil tests, to find out how 
far we can go with them. 

erJc 8 10 



In practice, we may develop relatively simple items requiring a small numbers of attributes, 
which are basically used for diagnosing the grade of attainment toward mastery of the at- 
tributes, and also more complex multi-attribute items that require dynamics of innovative use 
of the mastered or partially mastered attributes, etc. The resulting test will consist of such 
different types of questions. 



IV. Example 

We are in the process of collecting data, using a paper-and-pencil test on trigonometry for 
the first year high school students in Japan. At the moment, there are only 41 protocols. From 
these protocols, the strategy matrix (DiBello, Stout &; Roussos, 1993) has been made. 

TEST ITEMS 

There are 12 items in the test, and they are as shown below. They were taken from the 
original test of 17 items. Thus five items, (1)2., (2)2., (3)1., (4)1. and (4)4., .re left blank. 

(1) Obtain the values of the following. 

1. sin 90° + cos 180° + tan 45° 
2. 

3. sin [-f-] cos [-£-tt] - cos [-£-] sin [-§-*] 

(2) Simplify the following. 



1. (sin0 + cos0) 2 + 
2. 



(l-tang) 2 
1 + tan 2 0 



(3) Obtain the following when 0 is an angle in the third quadrant and sin 0 cos 0 = -y- . 

1. 

2. sin 0 + cos0 

(4) Solve the following trigonometric equation and inequality when x is as given 
„ in the brackets. 

1. 

2. 2 cos x + y/5 < 0 (0° < x < 360°) 

3. 2 sin 2 x + cos x -2 = 0 (0 < x < 2tt) 
4. 

(5) Suppose that AB = 5 , BC = 8 and B = 60° in the triangle ABC. 



1. Obtain the length of AC. 



2. Obtain sin A . 

3. Obtain the radius of the circumscribed circle. 

4. Obtain the area of the triangle ABC. 

(6) In the triangle ABC, we set BC = a and CA ~ 6 . When 

a 2 cos A sin B = 6 2 sin A cos/? , what shape does the triangle ABC have? 
Explain it in a concrete way. 

(7) In the triangle ABC, suppose that sin A : sin B : sinC = 7 : 8 : 13. Obtain 
the largest angle. 

Since this test is a part of the term examination, and the main objective of the exami- 
nation was to find out the students' achievement in trigonometry after a quarter of learning, 
most questions require orthodox approaches based on the mastery of attributes, A few items, 
including (6), may require some dynamics, however. The expected correct answers to these 
twelve items and the item score matrix are presented in Appendices 1 and 2, respectively. 

ATTRIBUTES AND STRATEGY MATRIX 

The attributes required in solving these trigonometry problems are stratified, in the sense 
that mastery of an attribute in each phase requires mastery of some or all attributes in the 
previous phases. 

1. Phase 1: prerequisites 

In this phase, easy tasks, such as addition and multiplication involving negative num- 
bers, fractions, etc., are excluded, for all examinees seem to have mastered them. 

(a) inequalities 

(b) factorization 

(c) distribution 

(d) Pythagoras theorem 

2. Phase 2: understanding of basic trigonometr y . . 

grade 1: sine, cosine, tangent on the triangle level, all positive values. 

grade 2: the above, but inclusive of negative values. 

grade 3: sine, cosine, tangent on the unit circle level, using radians. 

3. Phase 3: laws and theorems 

(a) area of a triangle using trigonometry 



(b) sin 2 6 + cos 2 6= 1 

(<0 tan « = [-SH-] 

(d) In a triangle, a side facing to a larger angle is longer than a side facing to a smaller 
angle. 

(e) law of cosines: c 2 = a 2 + b 2 — 2ab cos C 

(f ) law of sines: 



grade 2: 







t * i 


= f c 1 


[sin A. 


= 


.sinB. 


[sinC. 






6 


- f c 1 


[sin /I, 




sin B 


LsinC. 



NOTE: a, b, c: sides; A, B, C: angles; R: radius of the circumscribed circle; 
S: area 



The strategy matrix of these items is provided by Table 2. In addition to the attributes, a 
column is assigned for ability to restructure attributes and use them innovatively, etc., under 
the heading, dynamics. When the grade of attainment is considered for an attribute, the grade 
required for solving the item is written in the column; when only mastery and non-mastery 
are used, * is given, meaning that mastery of the attribute is required for solving the item. 
This table will become the most important information source when the pattern matrix is 
determined in factor rotation for defining the competency space 0 , differentiating 0^ from 
0 o . 



Insert Table 2 About Here 



V. Mathematical Models 

Here we only discuss mathematical models in the unidimensional latent space, although 
many models suitable for cognitive assessment can be easily expanded to the multidimensional 
latent space. Let 6 be the latent trait, which assumes any real number. Hereafter, all those 
functions observed in the multidimensional latent space earlier in this paper will be transfered 
to the corresponding functions of 0 . 

Samejima (1972) has proposed a general theoretical framework of the graded response model, 
in which the homogeneous case is distinguished from the heterogeneous case. The general 
graded response model represents a family of mathematical models which deal with ordered 
polychotomous categories in general. These ordered categories include: A, B, C, D and F in 
the evaluation of students' performance, strongly disagree, disagree, agree and strongly agree 
in a social attitude survey, partial credit given in accordance with the individual's degree of 
attainment toward the solution of a problem, to give some examples. 
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CRITERIA FOR EVALUATING MODELS 

The general graded response model includes many specific mathematical models. In an effort 
to select a right model, or models, for a specific psychological reality, the following features will 
be considered as desirable. 

1. The principle behind the model and the set of accompanied assumptions agree with 
the psychological reality in question. 

This is by far the most important criterion. 

2. The model provides additivity in the operating characteristics of the item scores x g 's 
(Samejima, 1994c). 

Additivity holds if the operating characteristics belong to the same mathematical 
model under finer recategorizations and combinings of two or more categories together. 

This implies that the unique maximum condition is satisfied by the resulting operating 
characteristics, if it is satisfied by those of the original x g 's . 

Graded item scores, or partial credits, are more or less incidental For example, it is 
a general practice to reevaluate the grades, A, B, C, D, and F, in a required course to 
pass and fail. 

Also, with the advancement of computer technologies, it is quite possible to obtain 
more abundant information from the individual's performance in computerized ex- 
periments as we proceed in research, and thus we need finer recategorizations of the 
whole cognitive process. 

3. The model can be naturally generalized to a continuous response model This criterion 
is a natural extension of additivity. 

4. The model satisfies the unique maximum condition (Samejima, 1969, 1972). 

Satisfaction of this condition assures that the likelihood function of any response 
pattern consisting of such response categories has a unique local or terminal maximum. 

5. The model provides the ordered modal points of the operating characteristics in ac- 
cordance with the item scores. 

The acceleration model introduced here is a model constructed with these considerations in 
mind. It is obvious that the first two criteria are most important for cognitive assessment. 

ACCELERATION MODEL 

The acceleration model is a family of models that belongs to the heterogeneous case, and is 
specifically appropriate in cognitive assessment 

ER|[C 12 |4 



Consider a as problem solving that requires a number of subprocesses before attaining the 
solution. It is assumed that there are more than one step in the whole process which are 
observable. Graded item scores, or partial credits, 1 through m g , are assigned to the 
successful completions of these separate observable steps. The processing function for each 
x 9 (= 1,2, ...,m p ) is given by 

where £ Xff (> 0) is the step acceleration parameter. 

It is assumed that the whole process leading to the solution consists of a finite number of 
clusters, each containing one or more steps, and within each cluster the parameters in ty Xg {Q) 
are common. Thus, if two or more adjacent x g 's belong to the same cluster, then the 
parameters in #x ff (#) are the same for these x g 's , and, otherwise, at least one of the 
parameters is different. 

Let w denote a subprocess, which is the smallest unit in the cognitive process. Thus each 
step contains one or more w 's . Let £ w (> 0) be the subprocess acceleration parameter, and 
then the step acceleration parameter, £ Xg , for each of x g = 1, 2, ...,ra 5 is given as the sum of 
£ w 's over all w £ x g . Thus we can rewrite the processing function in the form: 

M Xg (0) = [<m*)1 i: " 6 * ,{ " • 

The name, acceleration parameter, comes from the fact that, within each step, separate sub- 
processes contribute to accelerate the value of 8 at which the discrimination power is maximal 
to its ultimate position (cf. Samejima, 1994c). 

The acceleration model represents a family of models in which ty Xg (6) is specified by 
a strictly increasing, five times differentiate function of 0 with zero and unity as its two 
asymptotes, and 

[ft **M 2 

decreases with 0 . Thus the cumulative operating characteristic, P* g (0) , is given by 

p:,(<>) = n . 

u=0 

and we obtain for the operating characteristic 

= ft [*.(*)]*■ [i-^cn+dWI*"* 1 ] • 

If our experimental setting is improved and allows to observe the individual's performance in 
more finely graded steps, then m g will become larger. It is obvious (Samejima, 1994c) that the 
resulting operating characteristics still belong to the acceleration model: a partial satisfaction 
of the additivity criterion. It is also obvious that the model can be generalized to a continuous 



response model as the limiting situation in which there are infinitely many subprocesses in each 
step. 

Here a specific model in this family will be introduced, in which ^ Xg (9) is given by 

* X,W = 1+exp [-D a Xg (0 - /3 Xg )} ' ^ 

where D = 1.7 , and a Xp (> 0) and p Xq are the discrimination and location parameters, 
respectively. Figure 4 illustrates the six operating characteristics with m g = 5 and the 
parameters a Xg = 1.36517 , L03244 , 0.87524 ,. 1.09083 , 0.58824 , /3 Xg = -0.94260 , 
-0.76985 , 0.03941 , 1.35406 , 0.80000 , and £ Xg = 0.41972 , 0.51741 , 0.54196 , 0,60004 , 
1.00000 , for x g = i, 2, 3,4,5 . 



Insert Figure 4 About Here 



Since we have 

Q, - Or- 1 ^ 

" dmax [l+t x§ \ ' 

where 9d ma x indicates the value of 9 at which the processing function M x (0) is steepest, or 
most discriminating, it is obvious that 0dma X is a strictly increasing function of ( Xg . Applying 
this for the subprocess acceleration parameter £ w , we can say that within each step separate 
subprocesses contribute to accelerate 9d max to its ultimate position. 

It has been shown (Samejima, 1994c) that the unique maximum condition is satisfied in this 
model. It has also been shown (Samejima, 1994c) that the orderliness of the modal points of 
the operating characteristics practically holds in this model, except for very unusual cases. 

PARAMETER ESTIMATION 

In cognitive assessment, it is suggested to follow the procedure described below in parameter 
estimation. 

1. Use a nonparametric estimation method like Levine's (1984) or Samejima's (1983, 
1993b, 1994b), and estimate the operating characteristics, P Xg (9) 's . 

2. Tentatively parameterize the results using a very general semi parametric method, such 
as Ramsay and Wong's (1993), in which not only the fit of P Xg {9) but also that of 
its first partical derivative with respect to 9 are considered. 

3. From these results obtain the estimated processing function M Xg (9) and its partial 
derivative with respect to 9 . 

4. Select three arbitrary probabilities, p x , p 2 and p 3 , which are in an ascending order, 
and find out 0 X , 9 2 and 9 3 , at which M Xg (9) equals p x , p 2 and p 3 , respectively. 
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(2) 



(3) 



A 

5. The estimated acceleration parameter £ Xg is obtained as the solution of 

0 3 - 6 2 = logKpQ-^-lj-logKpa)- 1 ^ -1] 

~ 01 log [( Pl •)- 1/< '« " 1] ~ log [( P 2 - 1] • 

6. Obtain the estimate, f} x , as the solution of 

r 1 

*..(AJ = [yJ • 

7. From these results obtain the estimate of a Xg by 

= ~nT 86 M " {6) at 6 = Px ° ■ (4) 

Note that this method can be applied for any curve as long as ~ M Xg {6) is available. 
Suppose, for example, a model based on the individual choice behavior has been used, and 
then the researcher decides to switch to this specific acceleration model for i J ie rest of re- 
search. The method just explained can be used directly. Figure 5 presents the operat- 
ing characteristics in Masters' (1982) partial credit model, using a Xg = 1,2,3,4,5,6 and 
(3 Xg = 1.0,2.0,3.0,3.5, 1.8, 1.0 in Bock's (1972) multinomial model represented by 

p m = exp[q, g e + fi k§ ] 

with kg replaced by x g for x g = 0,1,2,3,4,5 , respectively. 



Insert Figure 5 About Here 



In fact, the parameters in the acceleration model used in Figure 4 were obtained by the 
parameter estimation method just described, from the M Xg (6) 's in Masters' model with the 
above set of parameters, setting pi = 0.1 , p 2 = 0.5 and p 3 = 0.9 in (2). Compare Figure 5 
with Figure 4. These two sets of curves are practically indistinguishable! The same procedure 
was repeated by setting the values of p r , p 2 and p z to (0.2,0.5,0.8), (0.25,0.50,0.75), 
(0.3,0.5,0.7) and (0.3,0.6,0.7) , respectively, and the results turned out to be very similar. 

The similarity of the two sets of curves exemplifies the fact that the acceleration model 
provides varieties of different curves. This is a strength, for researchers who want to switch to 
the acceleration model can do that without reanalyzing the protocols following the acceleration 
model, if he/she uses the method just described. 

The item response information function (Samejima, 1973, 1993b) of the graded item response 
x g is defined by 

= E lo S M u {9) - ~ log [1 - M (xa+1) (&)] . 

ERIC is I? 



Once this function is evaluated, The item information function, is obtained as its conditional 
expectation, given 0 , that is, 

/,(*) = E[I Xg {9)\0) =£/,,(0)P„(0) . 

Xg 

The item information functions obtained in these two models proved to be fairly close to each 
other (cf. Samejima, 1994c), as was expected from the similarity between the two sets of 
operating characteristics presented in Figures 4 and 5. 

ROBUSTNESS OF THE MODEL 

The assumption that a single set of a Xg and f3 Xg exists within each step may be violated, 
especially when m g is small. Robustness of the acceleration model will handle this situation, 
however. Suppose that we did not know there were two clusters involved, and treated them 
as a single step, estimating the step parameters following this specific acceleration model, and, 
later, with the improvement of the experimental setting, they were disclosed as two separate 
steps which belonged to two different clusters. The results obtained by treating them as a 
single step still provides good approximations , showing the robustness of the model. 

Suppose, on the contrary, we need to combine two steps which do not belong to the same 
cluster. Note that the resulting combined step will not belong to the acceleration model. 
Using the method described earlier, we can approximate the operating characteristic of the 
combined category following the acceleration model, and the result usually provides a good 
approximation. 

Figure 6 illustrates six step processing functions, the first three of which belong to a cluster 
with a Xg = 1.0 and j3 Xg = —1.0 , and the second three to another with a Xg = 1.0 and 
f3 Xg = 1.0 , respectively, and the third parameters are £ Xg * = 0.5,1.0,1.5, for the three steps 
in each cluster. It is obvious that the operating characteristic of the combined category of any 
two adjacent steps still belongs to this specific acceleration model, except for the combination 
of x g = 2 and x g = 3 . The two more curves in the same figure are the product of M Xg {6) ' 
for x g = 2 and x g = 3 (solid line) and its approximation following the acceleration model (a 
dash and two dots repeated), respectively. 



Insert Figure 6 About Here 



Figure 7 presents the operating characteristics of the seven steps, 0 through 6 , plus the 
sxim of two operating characteristics for x g = 2 and x g = 3 (solid line), and the approximated 
operating characteristic (a dash and two dots repeated) obtained by fitting a single M Xg (0) to 
the product of the original M z (0) and M 4 (0) following this specific acceleration model. The 
Pi ,P2 ,P3 used in this approximation were 0.21109,0.48884,0.79446 , and the corresponding 
#1,02,03 were -0.3,0.5,1.4 . The resulting estimated parameters obtained through (2), (3) 
and (4) turned out to be: £ Xg = 1.11338 , (3 Xg = 0.43006 , and a Xg = 0.86888 . 




Insert Figure 7 About Here 



The two curves for the combined category in Figure 7 overlap almost completely, showing 
the robustness of the model. Thus Additivity of the operating characteristics practically holds 
for this model. 

The reasons for this robustness of this model comes from the fact that the two parameters, 
a Xg and £ Xg , work compensatorily to determine the steepness of M Xg (6) , while £ x alone 
accounts for the shape of the curve. Thus a set of a large a Xg and a small £ Xg will provide 
the steepness of the curve similar to the one resulted from a set of a small a Xg and a large 
£ Xg . The shape of the curve is largely determined by £ Xg , however, as we can see in the earlier 
observation that Odmax changes as a function of £ Xg , thus together providing various shapes 
and steepnesses. 

VI. Discussion 

It is necessary to expand the acceleration model to the multidimensional latent space, in 
order to use it effectively in the competency space approach. It can be done in the same way 
the nomal ogive model on the continuous response level was expanded to the multidimensional 
latent space (Samejima, 1974). Several other directions of expansion are conceivable, including 
both compensatory and noncompensatory directions. 

A slower, patient approach may be more productive, however, attempting to identify Q b 
of a relatively small dimensionality first. It is expected in the example given here that Q b will 
be unidimensional, for the population of individuals is relatively homogeneous and the problem 
solving is limited within trigonometry of the high school level in Japan. Once a uridimensional 
Q b has been identified, then the acceleration model can be tested to find out how well, or 
badly, it works. 

The competency approach can be more effectively used in research that belong to category 2, 
in which an appropriate software for intensive observation of the individual's problem solving 
beh avior is developed and used. The only reason that such research has not been done yet 
is that it takes a sizable amount of research money. It is the author's hope that both the 
competency space approach and the acceleration model will be used by other researchers in 
varieties of different situations and feedback information will be given in the near future, so 
that the approach and the model can further be developed in useful directions. 
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APPENDIX 1 



Correct Answers to the Twelve Test Items 
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(1) Obtain the values of the following. 
1 . sin 90° + cos 1 80° + tan 45° 

= 1 + (—1) + 1 = 1 
3. sin [-=-] cos [-fir] - cos [-§-] sin [-fir] 

— ( n/3 N _ ( J_\ J_ _ 3 J_ 

2 V 2/ v 2 / 2 — "4 4 

(2) Simplify the following. 



1. (sin 0 + cos Of + 



(1-tanfl) 2 
l+tan 2 0 



1 + 2sin0 cosfl + 



1-2 tanfl-f tan 2 fl 
I 



cos 2 0 

= 1 + 2 sin 0 cos 0 + cos 2 0 - 2 sin 0 cos 0 + sin 2 0 
=1+1=2 

(3) Obtain the following when 0 is an angle in the third quadrant and 
sin# cos# = -f . 

2. sin 0 + cos 0 

(sin 0 + cos Of = 1 + 2sin 0 cos 0 = 1 + 1 = 2 

sin# + cos0 = —>/2 

(4) Solve the following trigonometric equation and inequality when x is as given 
in the brackets. 

2. 2cosz + \/3 < 0 (0°<2< 360°) 

cos x < — 

150° < 2 < 210° 

3. 2 sin 2 x + cos a; - 2 = 0 . (0 < x < 2ir) 

2(1 -cos 2 2) + cos x - 2 = 0 

2 cos 2 x — COS 2 = 0 

COS 2 (2 COS 2 — 1 ) = 0 

2 = — — 7T — — 7T 

(5) Suppose that AB = 5 , BC = 8 and B = 60° in the triangle ABC. 
1. Obtain the length of AC. 



20 



22 



6 2 = a 2 + c 2 - 2ac cos B = 8 2 + 5 2 - 2 • 8 • 5 • -§- 
= 64 + 25 - 40 = 49 



6 = 7 



2. 



Obtain sin A . 



sin A = [-f-] sin B 



~7 2 - 7 V<3 



3. 



Obtain the radius of the circumscribed circle. 




= 2R 



R = 




4. Obtain the area of the triangle ABC. 

S = [a • csinB] = • 8 • 5 • = 10v/3 

(6) In the triangle ABC, we set BC = a and C^4 = 6 . When 
a 2 cos A sin i? = 6 2 sin A cos Z? , what shape does the triangle ABC have? 
Explain it concretely. 



26c 




sin A 



sin B 






a 2 {b 2 + c 2 - a 2 ) = b 2 {a 2 + c 2 - b 2 ) 
a 2 c 2 - a 4 = b 2 c 2 - b 4 
c\a 2 - b 2 ) - (a 2 + 6 2 )(a 2 - b 2 ) = 0 
(a 2 - 6 2 )(c 2 - a 2 - b 2 ) = 0 
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2 — b 2 = 0 — > a = 6 isosceles triangle 



or 



c 2 - a 2 - 6 2 = 0 



c 2 = a 2 + 6 2 right triangle 

(7) In the triangle ABC, suppose that sin A : sin B : sin C = 7 
Obtain the largest angle. 

[_£_] = [_L_1 = [_£_] 

L sin .4 J [ sinS J I sin C J 

<z = 7fc, 6 = 8/c, c = 13/b 
C is the largest angle. 
c 2 = a 2 + 6 2 - 2a£> cos C 



cos C = 
C = 120° 



a 2 +fe 2 -c 2 



2a6 
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